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Weighted  Least  Squares  Estimators  on  the  Frequency  Domain 
for  the  Parameters  of  a  Time  Series 

Abstract 

A  procedure  for  estimating  the  parameters  of  a  time  series  is  proposed.  The  estimate  minimizes  a 
criterion  function  which  is  the  weighted  sum  of  squares  of  the  distances  between  the  periodograms 
and  the  spectrum  of  the  series.  Under  mild  conditions,  the  estimate  is  shown  to  be  strongly  con¬ 
sistent.  The  asymptotic  distribution  of  the  estimate  is  also  obtained.  With  a  proper  choice  of 
weighting  function,  the  estimate  has  the  same  asymptotic  distribution  as  the  one  for  the  max¬ 
imum  likelihood  estimate.  Simulations  were  carried  out  to  evaluate  the  performance  of  the  esti¬ 
mate. 

1.  Introduction 

We  consider  a  stationary  time  series  X(<),  <—0,1,  .  .  .  ,  T-l,  with  mean  zero  and  autoco¬ 
variance  function  c(«)  =  E[X(<)X(<  +  u)].  When  the  autocovariance  function  satisfies 

00 

£  I «(«)!  <  °°  (1.1) 

*=-00 

the  spectrum  density  function  of  the  series  X(<)  is  defined  by 

OO 

/(X)  =  (2w)_1  J2  e(tt)exp(-i'Xtt)  .  (1.2) 

«=-oo 

This  relation  can  be  inverted  to  obtain  the  representation 

c(«)  =  J'^exp(iuX)/(X)dX  .  (1.3) 

In  this  paper,  we  are  interested  in  the  situations  in  which  the  spectrum  of  the  series  depends  on 
some  unknown  parameters.  Here  are  some  examples: 
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1  J  [1+(X/X0)]2 

(1.4) 

M  ’  [l+(X/X0)2]3/2 

(1.5) 

/  ( X )  =0$  exp(-X2/3f2/2) 

(1.6) 

/  (x)=/32exp(- 1 X  |  //9j) 

(1.7) 

/(xH-^rexpHVx4) 

(1.8) 

«t2  ll+ije'V  •  •  •  +6?e,?x|2 

2tt  |  i+ a  iC  »>>-(-  •  •  ■  -f  op  e ,pX  | 2 

(1.9) 

Models  (1.4)  and  (1.5)  were  used  in  Aki  (1967)  to  model  the  spectra  of  far  field  body-wave  dis¬ 
placements  of  earthquakes.  The  parameter  X0  is  the  corner  frequency  and  Q0  is  proportional  to 
seismic  moment.  Seismologists  are  very  interested  in  estimating  the  parameters  X0  and  fi0,  which 
are  important  in  the  study  of  the  source  properties  of  earthquakes  (cf  Aki  and  Richards  (1980)). 
Slutsky  (1937)  used  model  (1.6)  in  studies  of  economic  time  series.  Model  (1.7)  has  been  used  by 
Lumley  and  Panofsky  (1964)  for  atmospheric  turbulence,  and  by  Whittle  (1962)  for  agricultural 
spatial  series.  Pierson  and  Moskowitz  (1964)  proposed  (1.8)  as  the  spectrum  of  ocean  waves 
caused  by  winds.  The  last  example  (1.9)  is  the  spectrum  of  an  autoregressive  moving  average  pro¬ 
cess  of  order  (p ,q)  (ARMA(p ,q)).  In  the  following  discussion,  we  let  / (X,0O)  represent  the  spec¬ 
trum  of  the  series  AT(f),  and  we  are  interested  in  estimating  the  parameter  90. 

The  criterion  function  used  in  this  paper  is  based  on  periodograms.  The  periodogram  at  fre¬ 
quency  X  of  a  series  X(t)  is  defined  as 

^)='^jrdx(^)dx(-^)  (1.10) 

where  d^(X)  is  the  finite  Fourier  transform  of  the  series  X(t), 

dx(^)=Y]X(t)exp(-i\t).  (1.11) 

t= o 

The  motivation  of  the  frequency  domain  approaches  is  from  the  observation  that,  under  some 
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mild  conditions,  the  periodograms  of  a  series  X(t)  on  the  Fourier  frequencies  0<X,.=2jr«/ T<7r 
are  asymptotically  independently  distributed  according  to  exponential  distribution  with  mean 
/(X,)  (cf  Brillinger  (1975)).  Therefore,  an  “approximate  maximum  likelihood  estimator”  can  be 
obtained  by  finding  the  0  that  maximizes  the  function 

m  =  -£log(/(X,0))  -  £/(X)//(X,0)  (1.12) 

X  X 

where  the  summation  is  over  the  Fourier  frequencies  in  (— 7r,7r)— (0).  Unless  indicated  otherwise,  we 
use  this  convention  throughout  this  paper.  When  the  spectrum  can  be  written  as 

/(X)=(2i)-1J(X,D0K,  (1.13) 

?(X,0)  does  not  depend  on  a2 .  We  could  obtain  an  estimate  of  60  by  minimizing 

L'(0)  =  £/(X)MX,0)  (1.14) 

x 

and  estimate  <Tq  by  o2=L,{0)/(2nT).  The  estimates  which  maximize  L{6 )  or  L'{0 )  were  studied  in 
Bloomfield  (1973),  Hannon  (1973),  Whittle  (1961)  Dzhaparidze  (1974),  Davies  (1973),  Robinson 
(1978)  and  Ibragimov  (1967).  Under  various  conditions,  the  estimates  were  shown  to  be  asymp¬ 
totically  consistent  and  efficient.  A  similar  estimation  procedure  was  proposed  by  Taniguchi 
(1981),  who  considered  the  estimates  which  minimize  the  distance  between  the  spectrum  and  the 
smoothed  periodograms. 

d{^,0)  can  be  viewed  as  the  periodograms  of  the  series  past  a  filter  whose  transfer  func¬ 
tion  A(X)  satisfies  |  A(X) 1 2=1/ g(\,0).  The  problem  of  minimizing  L'(0)  is  equivalent  to  finding 
j(X,0)  to  make  the  expected  value  of  I(\)/ g{\,0)  be  constant;  this  is  asymptotically  equivalent  to 
finding  a  filter  such  that  the  filtered  series  is  a  white  noise.  Therefore,  for  ARMA  processes,  the 
estimate  which  minimizes  L'(0)  is  asymptotically  equal  to  the  least  squares  estimate  discussed  in 
Box  and  Jenkins  (1970).  However,  the  function  L'{0 )  is  much  easier  to  handle  than  most  of  the 
objective  functions  defined  on  the  time  domain,  such  as  in  Box  and  Jenkins  (1970).  The  difficulty 
in  estimating  the  parameters  on  the  time  domain  is  mainly  due  to  the  intractability  of  the  max¬ 
imum  likelihood  equations  (even  for  Gaussian  processes). 
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In  the  following  sections,  we  study  the  properties  of  the  weighted  least  squares  estimate 
which  minimizes  the  criterion  function 

Qr(*)  =  £*(X)[/(MH(X)]2-  (1.15) 

X 

We  will  show  that,  with  a  proper  weighting  function,  the  estimate  has  the  same  covariance  matrix 
as  that  of  the  approximate  maximum  likelihood  estimate. 

The  weighted  least  squares  estimate  has  several  advantages:  (l)  Due  to  the  availability  of 
various  least  squares  algorithms,  it  is  easy  to  implement  the  estimation  procedure.  (2)  We  are 
often  only  interested  in  the  properties  of  the  processes  in  certain  frequency  bands;  by  using  a 
proper  weighting  function,  we  could  avoid  the  effects  from  other  bands  or  reduce  the  effect  of 
additive  noise.  (3)  The  approximate  maximum  likelihood  procedure  cannot  deal  with  the  cases 
where  /  ( \,90 )  vanishes  at  some  X.  The  weighted  least  squares  procedure  does  not  have  this 
difficulty.  (4).  Analogous  to  the  robust  procedures  for  independent  observations,  we  could  get 
estimates  resistant  to  the  presence  of  peaks  (corresponding  to  periodic  components,  such  as  sea¬ 
sonal  effect)  in  the  series.  Therefore,  it  is  possible  to  estimate  the  spectrum  of  the  de-seasoned 
series  without  removing  the  trend  from  the  series. 

2.  Assumptions  and  Results 

We  are  interested  in  the  series  X(t),  <=0,1,  .  .  .  ,  T- 1,  which  satisfies  the  following  assump¬ 
tion. 

Assumption  1.  X(t)  is  a  stationary  series  with  cumulants 

c*(«i,«2,  •  •  •  ,«*-i)  =  cum{X(<-|-«1),  .  .  .  (2.1) 

The  cumulants  satisfy 

00 

E  (1+ l«y  1)1  •  •  •  ,«*-i)|  <  oo  (2.2) 

for  j= 1,  .  .  .  ,  A-l  and  A=2,3,.... 


Under  Assumption  1,  X(<)  has  A-th  order  spectrum  /*(Xj,  .  .  .  ,X*_j)  with  bounded  and  uniformly 
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continuous  derivatives  (cf  Brillinger  (1975)). 

Since  the  second-order  properties  of  the  series  are  often  expressed  in  terms  of  its  spectrum, 
it  is  more  convenient  to  check  whether  ^[l-t-|u  |]|e(«)|  <oo  or  not  by  inspecting  the  spec¬ 
trum.  A  sufficient  condition  is  given  in  the  following  theorem.  The  proofs  of  this  and  other 
theorems  are  given  is  Section  4. 

Theorem  1.  Suppose  X(t)  is  stationary  and  has  spectrum  /(X)  with  a  bounded  and  uniformly 
continuous  derivative  /  ((X)  which  satisfies  a  Lipschitz  condition  of  order  a>0.  Then 
Jj[l+  |  «  |  ]  |  c(«)  j  <oo,  here  c(«)=c2(«)  is  the  autocovariance  function  of  the  series. 

All  of  the  examples  except  (1.7)  given  in  Section  1  satisfy  the  conditions  of  Theorem  1.  The  spec¬ 
trum  of  example  (1.7)  is  not  differentiable  at  X— 0,  and  the  autocovariance  function  is  of  order  u~2 
(see  Robinson  (1978)  and  Whittle  (1962));  thus  Assumption  1  is  not  satisfied  either.  For  situations 
similar  to  (1.7),  we  could  filter  the  series  to  make  the  spectrum  of  the  filtered  series  satisfy  the 
conditions  in  Theorem  1.  That  is,  we  can  use  filter  to  smooth  the  singular  points  of  the  spectrum. 
Of  course,  we  are  not  able  to  do  this  when  the  singular  points  depend  on  unknown  parameters. 

We  now  establish  two  theorems  which  are  of  some  independent  interest  and  are  more  gen¬ 
eral  than  the  results  required  to  prove  the  asymptotic  properties  of  the  weighted  least  squares 
estimates. 

Theorem  2.  Let  V'(X)  be  a  continuous  function  on  [— 7r,7r] .  If  X(t)  satisfies  Assumption  1,  then 

lim  -j^y )V,(X)/*(X)  =  ■~-ft/>(X)/k(X)dX  almost  surely,  (2.3) 

J  x  2tt 

for  k>l.  Here  /(X)  is  the  (second  order  cumulant)  spectrum  of  X(t). 

Theorem  2  can  be  easily  extended  to  the  following  result. 

Corollary  1.  Suppose  is  uniformly  continuous  on  (X,0)  E  [— 7r,7r]  X0,  and  X(t)  satisfies 

Assumption  1.  Then,  for  £  >1, 

lim  almost  surely, 

1  x  ™ 


(2.4) 
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and  the  convergence  is  uniform  over  ©. 

The  asymptotic  distributions  of  the  variables  in  Theorem  2  are  given  below. 


Theorem  3.  Under  the  conditions  in  Theorem  2, 


yfT 


^EV’(x)/*(x)-^/>(x)/*(x)dx 


2  ff ' 


N(  (W) 


(2.5) 


where 


al  =  (2^)-1A:2(2A;-2)!/ V(X)/2*(X)dX  +  (2jt)-1*2(2A:-2)!  f  ’V(XM-X)/2*(X)dX  (2.6) 

*-T  "-T 

+  (2;r)-1*2(i!)2/J/Jv<X1)^X2)/4(X1)-X1)X2)/*-1(X1)/*-1(X2)dX1dX2  . 

Remark  1.  For  series  with  /4(X1,X2,X3)=0,  such  as  in  the  case  of  Gaussian  processes,  the  third 
term  of  (2.6)  vanishes,  and  the  asymptotic  distribution  is  the  same  as  the  one  in  which  /(X,)  are 
true  independent  variables. 

In  proving  the  theorem  concerning  the  strong  convergence  of  the  weighted  least  squares  esti¬ 
mates,  we  need  assumptions  about  ©,  f(\,0),  and  the  weighting  function  </>(X). 


Assumption  2.  f(\,0)  is  continuous  on  (X,0)  6  [-jr,jrjx©,  and  ©  is  a  compact  set  in  Rr . 


Assumption  3.  is  a  symmetric  continuous  function  on  [— ?r, 7r]  and  for  all  0y^0o, 

jV(X)[/(X,0)-/(X,0o)]2  dX>0. 

Assumption  3  requires  that,  on  the  support  of  </>(X),  f{\,0o)  is  different  from  f{\,0),  0y^9o.  From 
Corollary  1  and  Theorem  2,  the  strong  consistence  of  the  weighted  estimate  can  be  established. 

Theorem  4 ■  Under  Assumptions  1  to  3,  the  estimate  0  which  minimizes  Qt{9)  of  (1.15)  converges 
to  0O  almost  surely. 

We  need  some  more  assumptions  to  derive  the  asymptotic  distribution  of  the  weighted  least 
squares  estimate 


Assumption  4 ■  &o  is  an  interior  point  of  ©. 


-  7  - 


Assumption  5.  In  a  neighborhood  of  90,  /(X,0)  is  twice  differentiable  with  respect  to  6,  and  the 
derivatives  are  continuous  in  X  and  6. 

Assumption  6.  The  r  by  r  matrix  A=A[0o)  is  positive  definite,  and  the  ij -th  element  of  A{6)  is 

(2.7) 

where  /,-(X,0)  is  the  derivative  of  /(X,0)  with  respect  to  0,-,  t=l,  .  .  .  ,r. 

We  now  describe  the  asymptotic  distribution  of  the  weighted  least  squares  estimate. 

Theorem  5.  Under  the  conditions  in  Theorem  4,  and  Assumptions  4  to  6,  let  9  be  the  estimate 
which  minimizes  Qr(0)  of  (1.15).  Then  \/T  (0-Oo)  is  asymptotically  normal  with  mean  zero  and 
covariance  matrix  2A~lBA~l+A~1DA~1.  Here  A  is  given  in  Assumption  6.  The  «/-th  element  of  B 
and  D  are,  respectively, 

Hf-^/AX)AX^(XA)/A«dX  (2-8) 

and 

4  =  ^J/^(^)^)/4(^-X^)/.(X^o)/y(X,0o)dX  .  (2.9) 

Remark  2.  If  we  let  0(X)=/~2(X,0o),  we  obtain  the  same  covariance  matrix  as  the  one  in  Theorem 
5  of  Robinson  (1978),  which  is  the  asymptotic  covariance  matrix  of  the  estimate  which  maximizes 
the  approximate  likelihood  function  L(6)  of  (1.12) 

Remark  3.  For  Gaussian  processes,  /  4(X,p,w)=0,  the  matrix  D  vanishes,  and  when 
0(X)=/~2(X,0o),  the  covariance  matrix  is  2A_1,  which  is  the  asymptotic  covariance  matrix  of  the 
maximum  likelihood  estimate  (see  Hannon  (1973)).  Therefore,  the  estimate  with  weighting  func¬ 
tion  </>(X)=/~2(X,0o)  is  asymptotically  efficient. 

Remark  4.  An  efficient  estimate  can  be  obtained  by  using  an  iterative  procedure.  We  could  use 
the  unweighted  least  squares  estimate  to  start  with,  or,  use  the  weighted  function  <^>(X)=/~2(X). 
Here  /(X)  is  an  estimate  of  the  spectrum  obtained  by  smoothing  the  periodograms.  We  then  let 
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^.(x)=/-2(xA), 

where  0O  is  our  initial  estimate,  and  find  the  estimate  0l.  Usually,  only  two  or  three  iterations  are 
required. 

The  covariance  matrix  in  Theorem  5  depends  on  the  matrices  A ,  B  and  D .  A  and  B  can  be 
estimated  by  A (9)  and  B(6)  respectively.  An  estimate  for  the  matrix  D  is  suggested  by  the  fol¬ 
lowing  theorem. 

Theorem  6.  Let  ip(\)  be  a  continuous  function  on  and  let  X(t)  satisfy  Assumption  1,  then 

(2-10) 

1  x  „ 

converges,  in  meam  square,  to 

(2,r)‘1/  /  V<X)V>(/i)/4(X,-X,/ <)dXd/<  +  (2a-)-1/  V^(X)/2(X)dX  (2.11) 

+  (2^)-1/JV'(X)V>(-X)/2(X)dX  +  [(2^r1/V(X)/(X)dx]2  . 

3.  Simulation 

We  conducted  two  simulation  experiments  to  evaluate  the  performance  of  the  estimate.  We 
simulated  autoregressive  processes 

X(<)-0.6X(t-l)+0.3X(<-2)  =  e{t)  (3.1) 

where  e(t)  are  independent  random  variables  with  distribution  N{ 0,1).  The  size  of  the  simulated 
series  is  256.  The  sample  size  is  200  for  both  experiments. 

In  the  first  experiment,  we  use  the  weighted  function  <£(X)=/-2(X,0o)  to  find  the  estimate  0T. 
The  second  experiment  uses  an  iterative  procedure  to  find  the  estimates.  The  initial  estimate  90  is 
the  unweighted  least  squares  estimate.  The  first  iterative  estimate  6X  is  the  estimate  using  the 
weighting  function  <^(X)=/-2(X,0o)-  The  second  iterative  estimate  6%  is  the  estimate  using  the 
weighting  function  0(X)=/-2(X,^1).  Table  1  compares  the  sample  means  of  the  estimates  6T,  0O,  6 x 
and  $2  with  the  true  parameters.  In  Table  2,  the  sample  covariance  matrices  are  compared  with 
the  asymptotic  covariance  matrix  of  the  estimate  6 x,  which  is  also  the  asymptotic  covariance 
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matrix  of  the  maximum  likelihood  estimate.  From  Table  2,  it  can  be  seen  that  the  sample  covari¬ 
ance  matrix  of  the  second  iterative  estimate  02  is  very  close  to  the  asymptotic  covariance  matrix 
of  0„. 


4.  Proofs 

We  now  give  proofs  of  the  results  in  Section  2. 

Proof  of  Theorem  1.  From  integration  by  part,  we  have 

IT 

iuc(u)  =  i«J^/(X)exp(»«X)dX  (4.1) 

=  | /(;r)[exp(jW7r)-exp(-au7r)]j  -  J  /'(X)exp(au X)dX. 

Since  the  first  term  in  (4.1)  is  equal  to  zero,  then 

00  00  T 

E  I  «c(«)  I  =  E  |/_)r/'(X)exp(i«X)dX|  <  oo  (4.2) 

«=-oo  «=-oo 

from  Theorem  (3.6)  of  Zygmund  (1959)  (p.  241).  The  theorem  is  finished  by  noting  that  c(0)  is 
bounded. 


Proof  of  Theorem  2.  Let 


V>t(X)=  E  <?(«)exp(»«X) 


(4.3) 


> — L 


be  the  Cesaro  sum  of  the  Fourier  series  of  ^>(X),  and  let  Sy=EV’i(X)/*:(X).  The  expected  value 

\ 

of  /*(X)  is  equal  to 


(2;rr)-*E 


2k 


ndx(X.) 


1=1 


(2?r T)  *  ^cum(d^(X,):iGi/i)  *  *  *  cum(dJ(X,):«Gi/p)  (4.4) 


where  X,=(-1)'X,  and  the  summation  is  over  all  partitions  i/=iq[J  •  •  •  of  the  set 

(1,2,  ...  ,21:}.  From  Theorem  4.3.2  of  Brillinger  (1975), 


cumld^Xj),  .  .  .  ,  dx(X*)} 


(4.5) 


=  (27r)*-1A(T)(  EXy)  /*(X„  •  •  •  ,  X*_j)  +  0(1) 


where 
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A^r)(X)  —  ^exp(-j’Xl)  .  (4.6) 

t=o 

The  function  A^r^(X)  has  the  properties:  A^t\\)—T  for  X=0  (mod27r),  and  A^(2ns / T)=0  for  a 
an  integer  with  8f^0  (modT).  The  product  of  the  cumulants  in  (4.4)  is  of  order  Tk  for  the  parti¬ 
tions  consisting  of  k  sets,  each  of  which  contains  an  even  number  and  an  odd  number.  The 
number  of  such  partitions  is  k\,  and  the  products  of  the  cumulants  are  equal  to 

[cum(dx(X),dx(-X))]A  =  (2nT)k  fk(\)+0(Tk~1)  .  (4.7) 

For  other  partitions,  the  orders  of  the  products  are  smaller  than  Tk .  So, 

lim  1e[EV-l(X)/‘(X)]  =  ±L/_'<MX)/*(X)dX.  (4.8) 

In  the  proof  of  Theorem  3,  we  will  show  that  the  variance  of  T~1St  is  of  order  T~l.  This  implies, 
by  Chebyshev’s  inequality  and  the  Borel-Cantelli  Theorem,  that  N~2Sn2  converges  almost  surely 
to  the  right  hand  side  of  (4.8).  To  prove  the  theorem,  it  is  sufficient  to  show  that  SM  does  not 
differ  much  from  the  nearest  V 

Let  Bt  =  (2ir)*  T*_1[Sr +^>(0)7(0)],  and  note  that 

Bt  =  E  ?(«)E*(«iPa*i)  •  •  •  X(tk)X(sk)  (4.9) 

«=-L  qT 

where  the  second  summation  is  over  all  (<i,«i,  .  .  .  ,  tk,8k)  in 

ft«r={(<i,3i,  •  •  •  ,tk,*kY-  E(**~s»-)=tt  mod(T),  1  <ti,Si<T}.  (4.10) 

i=i 

Let 


gm=bm-bn2=  E  ?(«)  E  ^(tox^o  •  •  •  x(tk)x(*k).  (4.11) 

Then 


E GM  <  E  E  ?(«)?(v)  E  E 

«  =-Lv  =-L 


(4.12) 


nM-nN\iM-n»2 


E|x(fj)X(si)  •  •  •  X(4)X(St)X(<'1)X(,'1)  •  •  •  X(f'*)X(«'*)| . 


The  expectation  inside  the  summation  of  (4.12)  is  equal  to 
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][]cum(X(z,-);j€z'i)  •  '  '  cum(X(z,-);t6^p) 

V 

where  the  summation  is  over  all  partitions  of  {1,2,  .  .  .  ,4 k),  and 


(4.13) 


y=(t+l)/2,  i  is  odd  and  i<2k 

_  sji  j=i/2,i  is  even  and  i<2k 

z‘  ’  t'j,  j={i+\)/2-k,  i  is  odd  and  2k  <i 
s'j,  y==  i/2-k,  i  is  even  and  2k  <  i 


(4.14) 


The  partitions  giving  non-zero  products  must  consist  of  sets  with  two  or  more  items.  The  number 
of  sets  is  therefore  smaller  or  equal  to  2k.  After  rearranging  the  index  in  (4.12)  according  to  the 
partitions,  we  see  that  EG  fa  is  of  order  N2k .  Therefore, 


(N+ 1)2 

E[  max  GM}2<  £  EGJ&=0(iV2*+1). 

N2<M<{N+lf 


(4.15) 


Hence  ElmaxfX  2SM~N  2Sw2]2}  is  of  order  N 1  2*  for  k> 2.  For  Jfc=l,  it  can  be  shown  by  straight 
M 

calculation  that  the  expectation  is  of  order  N~2. 

Finally,  writing  T~1St  as 


1 

iv2  It  n2  n2  n2 


and  applying  Chebyshev’s  inequality  and  the  Borel-Cantelli  Theorem,  we  prove  that 


*! 


(4.16) 


lim  T~'St  =  ±-f^L(\)fk(\)d\ 


(4.17) 


almost  surely.  The  theorem  now  follows  from  the  uniform  convergence  of  il>L  (X)  to  ip(\)  (cf 

Edward  (1979),  p.  87)  and  the  almost  sure  convergence  of  T’~1]T]/*(X). 

X 


Proof  of  Corollary  1.  Let 


tMX,0)=  X)  ?(«,0)exp(*uX) 
%=-L 

be  the  Cesaro  sum  of  the  Fourier  series  of  ip(\,6).  Since 


(4.18) 


?(M>0)I^|  /  ^(X,0)exp(»Xw)dX  |  <2jt  suj>  |/(X,0)|. 


(4.19) 


q(u,6)  is  uniformly  bounded.  From  this  and  the  fact  that  xfjL(\,6)  is  a  linear  combination  of  the 
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2L+1  functions 

^(X,M)=jl-J-^-L|exp(»X«),  u  =-L ,  ,L,  (4.20) 

we  establish  the  uniform  convergence  of 

^E<MM)/*(X).  (4.21) 

x 

The  corollary  follows  from  the  uniform  convergence  of  to  rp(\,6),  which  can  be  shown  by 

slightly  modifying  the  proof  of  3.2.2  of  Edwards  (1979). 


Proof  of  Theorem  3.  The  variance  of  T  1,/2^i/j(X)/*(X)  is  equal  to 

x 

^1EEV’i(X1)V>L(X2)cum(/*(X1))/*(X2))  (4.22) 

\1  x2 

=  T-2*-1  (27r)-2* E E E^i- (X l) V’l (X2)cum {dx (w,y )',*) Et'i ) }  •  •  •  cum{d^(w,7);«.;'e^p)}, 

where  w,y=(-l)JX,-  and  the  summation  of  v  is  over  all  indecomposable  partitions  of  the  table  (cf 
Brillinger  (1975)) 


From  (4.5),  it  can  be  seen  that  there 
non-zero  limits: 


(4.23) 


(1.1)  ‘  '  (1.2*) 

(2,1)  •  •  •  (2,2 k) 

are  only  three  groups  of  indecomposable  partitions  giving 


(1) .  There  is  one  set  which  contains  two  items  from  each  row;  one  of  the  two  items  in  the  same 
row  is  from  an  odd  column  and  the  other  is  from  an  even  column.  Each  of  the  other  sets  contains 
two  items  (one  from  an  even  column,  the  other  from  an  odd  column)  from  the  same  row.  The 
number  of  this  kind  of  partition  is  fc2(A:!)2,  and  the  product  of  the  cumulants  is 

(2jr)2*+1r2*-1/4(X1,-X1,X2)[/(Xl)/(X2)]*-1,  (4.24) 

where  /4(X,/*,w)  is  the  fourth  cumulant  spectrum  of  X(t). 

(2) .  There  are  at  least  two  sets  which  contain  one  item  from  each  row;  both  items  are  from  even 
columns.  The  cumulants  of  these  two  sets  are  of  order  T  when  the  X,’s  satisfy  the  restraint 
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X  j=— X2.  The  other  sets  could  contain  two  items  from  the  same  row  (one  from  an  even  column 
and  the  other  from  an  odd  column)  or  from  the  different  rows  (must  satisfy  the  same  restraint). 
The  number  of  such  partitions  is  k\2k-2)\,  and  the  product  of  the  cumulants  is  (2i rT’)2*/2*(X1). 


(3)  Same  as  (2),  but  the  sets  consisting  of  items  from  different  rows  contain  one  item  from  an 
even  colum  and  one  item  from  an  odd  colum.  The  cumulants  of  these  two  sets  are  of  order  T 
when  the  X,  ’s  satisfy  the  restraint  X!=X2.  The  number  of  such  partitions  is  A2(2£-2)!,  and  the 
product  of  the  cumulants  is  (2jtT,)2*/2*(X1). 


Therefore  The  variance  of  T_1/2^]V'(X)/*(X)  is  equal  to 

x 

2^2(*!)2r-2EEV<x1)V<x2)/(x1,-x1,x2)/*-l(x1)/‘-1(x2)  (4.25) 

Xi  x2 

+  i2(2i-2)!r-1EV'2(X)/2*(X)  +  k2{2k-2)\T-l^{\)rJi-\)f^{\) 
x  x 

which  converges  to  the  erf  of  (2.6)  in  Theorem  3. 

We  next  prove  the  asymptotic  normality  by  showing  that  the  cumulants  of  order  bigger 
than  2  converge  to  zero.  The  Ath-order  cumulant  is 


cum{T-1/2EV'(X)/*(X))  .  .  .  ,  r-1/2X>(X)/*(X)  (4.26) 

xi 

=  T^~hk~h/2\2i{yhk E E X) (X i) Vy (X2)cum (dx(wy ); */' €iq) }  •  •  •  cum{d/(w,J);i/6i'p)}, 

v  Xj  X2 

where  w,-y=(-l)JX,-  and  the  summation  of  v  is  over  all  indecomposable  partitions  of  the  table 

(1,1)  •  '  •  (1,2*) 

(2,1)  •  •  •  (2,2*) 

(4.27) 

(A,l) •  •  ■  (A, 2*) 

For  any  indecomposable  partition,  we  consider 

At^  2) E I] (X l ) (X2)C U m { dj( w,-y ); ij 6^ ) }  •  •  •  cum{d^(wl7);jj 6^)}.  (4.28) 

v  X  |  Xo 


For  the  partition  v,  we  can  find  a  set  tq  which  contains  items  from  different  rows.  If  does  not 
contain  items  from  every  row,  we  can  find  another  set  which  contains  some  items  from  rows 
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different  from  the  rows  of  the  items  in  iq.  If  the  union  of  and  u2  does  not  contain  items  from 
all  rows,  we  can  find  a  set  us  which  contain  some  items  from  rows  different  from  the  rows  of  the 
items  in  or  v2.  We  repeat  this  until  U=ux\^j  •  •  •  (Jr/,  contains  items  from  every  row.  Sup¬ 
pose,  in  these  n  sets,  there  are  m  sets  whose  corresponding  function  does  not  introduce  linear 
restraints.  Each  of  these  m  sets  must  contain  at  least  4  items.  The  number  of  items  in  U  is  at 
least  h+n-l+2m  and  the  number  of  sets  of  the  partition  is  smaller  or  equal  to 
hk-(h  +  n-l+2m)/2+n .  Also,  the  number  of  linear  restraints  introduced  by  the  corresponding 
functions  of  the  remaining  is  n-m.  Therefore,  (4.28)  is  of  order  T  to  the  power 

[hk-(h+n-l+2m)/2+n]+[h-n+m]-hk-h  /2  = -(n-l)/2.  (4.29) 

Some  possibilities  for  the  u  with  n=l  are  as  follows: 


(1)  iq  contains  at  least  two  items  from  each  row.  Then,  the  number  of  sets  is  at  most  (k-l)h  f-1. 
Hence,  (4.28)  is  of  order  T  to  the  power  of  (k-l)h+l+h-kh-h /2— l-A/2. 


(2)  ux  contains  just  one  item  from  some  rows.  Then,  there  must  exist  a  set  i/2  which  contains 
items  from  different  rows.  The  restraint  introduced  by  v>2  is  different  from  the  restraint  introduced 
by  v\.  So,  (4.28)  is  of  order  T  to  the  power  [kh-h /2+l]+[h-2]-kh-h /2=-l. 


(3)  Same  as  (2),  but  the  restraint  introduced  by  u2  is  the  same  as  the  restraint  introduced  by  iq, 
the  number  of  items  in  iq^Jt/2  is  at  least  2 h  and  (4.28)  is  of  order  T  to  the  power 
[kh-h  +2]  +  [A-l]-AA-A  /2=1  -h  /2. 


Therefore,  the  Ath-order  cumulants  of  T‘_1//2]T]V’(X)/*(X)  (4.26)  converge  to  zero  for  all  A  >2  and 

x 


the  proof  is  finished. 


Proof  of  Theorem  4.  From  Theorem  2, 

lim^E^Wm  =  2(2*)-'/ V(X)/2(X,0o)dX  (4.30) 

1  \  "T 


almost  surely.  From  Corollary  1 
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lin4l>(X)/(X,0)I(X)  =  2(2  ff)-1/>(X)/(X,e)/(X,0o)dX  (4.31) 

1  x  * 

almost  surely  and  uniformly  on  ©.  Therefore 

limr-1^*)  =  (2^-1/V(X)[/2(X,0)-2/(X^)/(X,0o)+2/2(X^o)]dX  (4.32) 

—IT 

=  (27r)-1/_V(X)[/(X,eH(X^o)]2dX  +  {2^  jj{\)f\\,6Q)^  . 

=  Q{0) 

Under  Assumption  3,  the  minimum  of  Q{6)  is  attained  only  at  9=60.  Let  0T  be  a  sequence  of 
estimates  which  minimizes  Qt{9)-  Suppose  that  there  exists  a  subsequence  6xn  converging  to 

e'y^e0.  Then 

g(^)=lim^1gT(§rJ<limT-1Qr(^o)=Q(^o)  (4-32) 

which  is  a  contradiction,  and  the  proof  is  finished. 

Proof  of  Theorem  5.  This  follows  immediately  from  Theorem  3  and  Corollary  1  by  using  the  clas¬ 
sical  argument  in  Jennrich  (1969). 


Proof  of  Theorem  6.  The  expected  value  of  (2.10)  is  equal  to 


Since 


(27rT)-2T'2X;EV'(X)V'(^)E[d3[(X)dJ[(-X)dJ(/i)] 

X  n 


E[dx(X)dx(-X)dx(^)dx(-/*)]  =  cum{d^(X),dJ(-X),d/(/i),dJ(-;i)} 

+  cum{dx(X),dx(-X)}cum{dx(p)dx(-p)} 

1  - ‘{dx(X),djT(p)}cum{d^(-X),d bf(-m« )} 

{dx(X),dx(-^)}cum{dx(-X),dx  (/*)}, 


+  cum{ 
-I-  cum{ 


(4.34)  is  equal  to 

+  t-1e^(x)/2(x) 

X  ft  X 

r  i 


+ 


[r-‘E*x)/(x) 
X  X 


+  °(4)> 


(4.34) 


L  J 

which  converges  to  (2.11)  in  Theorem  5.  Using  a  similar  argument  in  the  proof  of  Theore 


can 


show  that  the  variance  of  (2.10)  is  of  order  T  *,  and  finish  the  proof. 
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Table  2 

(X10“3) 

Theoretic 

9t 

^0 

h 

^2 

Var(£2) 

7.81 

6.98 

25.23 

8.54 

8.18 

Var(a1) 

3.55 

3.43 

13.36 

3.68 

3.35 

Var(a2) 

3.55 

3.22 

8.40 

3.68 

3.36 

Cov(o,2,a1) 

0 

-0.34 

13.31 

-0.12 

0.54 

Co  v(«r2,a2) 

0 

0.41 

-9.33 

0.54 

-0.42 

Cov(ai,a2) 

-1.64 

-1.47 

-8.03 

-2.11 

-1.51 

Comparison  of  the  sample  covariance  matrices  of  the  estimates  to  the  asymptotic  covariance 
matrix  of  0T. 


